Live streaming using multiple content streams

ABSTRACT

Complexity requirements of the Live Video Streaming application are addressed by introducing spatial and temporal redundancy at the stream and networking layers and combining it with advanced reconstruction techniques at the client side. A system includes: a plurality of single or multiple content sources; weighted temporal redundancy generator; a network, coupling the single or multiple content sources and weighted temporal redundancy generator, wherein the network distributes data of the sources and generator; and a plurality of single or multiple content consuming entities, coupled to the network.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims the benefit of U.S. patent application61/720,873, filed Oct. 31, 2012, which is incorporated by referencealong with all other references cited in this application.

BACKGROUND OF THE INVENTION

The present invention relates to the field of video transmission, andmore specifically to a system for live video streaming through a data orcommunications network to a client.

Live video streaming is a difficult technical challenge due to latencyconstrains, communications network bandwidth and system robustness. Suchstreaming gains exponential complexity once multiple points of view orchannels require multiplexing. Examples are multiple Picture-in-Picture,Channel guide, and Multiuser Videoconferencing.

Low latency requirements and real-time content make the application ofretransmission protocols, large jitter buffers and pause andplay/buffering (trick play) techniques almost impossible. Complexcommunications network topology, with lack of control over intermediatenodes (e.g., the Internet), require a unidirectionally robust protocolwith low latency built-in into the Client-Server (Application-Cloud)solution.

Adding to this complexity is the need to deliver multiple contentstreams to multiple clients with per-client stream multiplexingpatterns. This typically calls for powerful video transcoding operationsat the server such as, but not limited to, MCU (multiconferencing unit)in case of videoconferencing. The role of such operations is to composea multiplexed image based on per-client requirements from the originalfull resolution streams arriving from multiple sources. This step allowsthe reduction of bandwidth and decoding requirements at the client whichwould otherwise be exposed to all the incoming full resolution streams.

For example, live concert streaming may present multiple camera views inthe client application showing main/selected camera content in fullresolution and additional cameras in low resolution preview mode forinteractive selection or dynamic switching between views. Clicking onthe preview channels switches to the full resolution stream from theselected source, while previous full resolution content falls back intopreview mode.

Content switching may be accomplished manually, via user control, orautomatically as in case of videoconferencing with multiple participantsand full resolution content selection based on voice activity (e.g.,active talker detection).

Another important property of live video streaming such as sportingevents for example, is that interruption of the content flow has a moredrastic impact on the user experience compared to loss of highresolution details. An example of this problem is a “freeze” of livestreaming of a soccer match in a family living room during a criticalshot, while all the neighbors are already screaming “goal!”

Therefore, what is needed is a new and improved way to deliver multiplecontent streams to multiple live video clients with per-client streammultiplexing patterns.

BRIEF SUMMARY OF THE INVENTION

The invention presented herein addresses the complexity requirements ofthe Live Video Streaming application by introducing spatial and temporalredundancy at the stream and communications networking layers andcombining it with advanced reconstruction techniques at the client side.

A system includes: (I) Single or multiple content sources which mayinclude but not limited to Broadcast Encoding Stations, contentgenerating portion of Video Conferencing clients, Transcoding Gatewaysand Pre-recorded content distribution systems further referred to asSOURCE. (II) Weighted temporal redundancy generator. Weighting propertyis classified as the ability to provide different level of errorprotection for the different parts of the processed content. (III)Communications network which may include but not is not limited to, oneor more of Wireline Service Provider Access Network, Wireless ServiceProvider Access Network, Internet Core Network, and Private and PublicNetwork, as the means of data distribution, further referred to asNETWORK. (IV) Single or multiple content consuming entities which mayinclude but not limited to Retransmission nodes, Transcoding Gateways,Video Conferencing Clients, Multimedia content players further referredto as CLIENT.

Other objects, features, and advantages of the present invention willbecome apparent upon consideration of the following detailed descriptionand the accompanying drawings, in which like reference designationsrepresent like features throughout the figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a simplified block diagram of a distributed data orcommunications network.

FIG. 2 shows a client or server system.

FIG. 3 shows a system block diagram of a client or server system.

FIG. 4 shows an example of a typical live video conferencing sessionclient screen.

FIG. 5 shows show an example of a typical multisource live newsstreaming client screen.

FIG. 6 shows a block diagram of the system elements and theirfunctionality.

FIG. 7 shows a block diagram of the content recovery and selection bythe client.

FIG. 8 shows a block diagram of a specific implementation of contentreconstruction.

FIG. 9 shows the internal operation of a typical state of the art videoconferencing MCU.

FIG. 10 shows a specific implementation of a video conferencing MCUoperation

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a simplified block diagram of a distributed data orcommunications network 100 which embodiment of the present invention canbe applied. Data or communications network 100 includes a number ofclient systems 113, 116, and 119, and server systems 122 coupled to acommunication network 124 via a plurality of communication links 128.Data or communication network 124 provides a mechanism for allowing thevarious components of distributed network 100 to communicate andexchange information with each other.

Data or communication network 124 may itself be comprised of manyinterconnected computer systems and communication links. Communicationlinks 128 may be DSL, Cable, Ethernet or other hardwire links, passiveor active optical links, 3G, 3.5G, 4G and other mobility, satellite orother wireless communications links, wave propagation links, or anyother mechanisms for communication of information. Various communicationprotocols may be used to facilitate communication between the varioussystems shown in FIG. 1. These communication protocols may include VLAN,MPLS, TCP/IP, Tunneling, HTTP protocols, wireless application protocol(WAP), vendor-specific protocols, customized protocols, and others.While in one embodiment, communication network 124 is the Internet, inother embodiments, communication network 124 may be any suitablecommunication network including a local area network (LAN), a wide areanetwork (WAN), a wireless network, a intranet, a private network, apublic network, a switched network, and combinations of these, and thelike.

Distributed communications network 100 in FIG. 1 is merely illustrativeof an embodiment incorporating the present invention and does not limitthe scope of the invention as recited in the claims. One of ordinaryskill in the art would recognize other variations, modifications, andalternatives. For example, more than one server system 122 may beconnected to communication network 124. As another example, a number ofclient systems 113, 116, and 119 may be coupled to communication network124 via an access provider (not shown) or via some other server system.

Client systems 113, 116, and 119 typically request information from aserver system which provides the information. For this reason, serversystems typically have more computing and storage capacity than clientsystems. However, a particular computer system may act as both as aclient or a server depending on whether the computer system isrequesting or providing information. Additionally, although aspects ofthe invention has been described using a client-server environment, itshould be apparent that the invention may also be embodied in astandalone computer system.

Server 122 is responsible for receiving information requests from clientsystems 113, 116, and 119, performing processing required to satisfy therequests, and for forwarding the results corresponding to the requestsback to the requesting client system. The processing required to satisfythe request may be performed by server system 122 or may alternativelybe delegated to other servers connected to communication network 124.

According to the teachings of the present invention, client systems 113,116, and 119 enable users to access and query information stored byserver system 122. In a specific embodiment, the client systems can runa standalone application such as a desktop application or mobilesmartphone or tablet application. In another embodiment, a “web browser”application executing on a client system enables users to select,access, retrieve, or query information stored by server system 122.Examples of web browsers include the Internet Explorer browser programprovided by Microsoft Corporation, Chrome browser program provided byGoogle Inc., Safari browser program provided by Apple Inc., and theFirefox browser provided by Mozilla, and others.

In a client-server environment, some resources (e.g., files, music,video) are stored at the client while others are stored or deliveredfrom elsewhere in the network, such as a server, and accessible via thenetwork (e.g., the Internet). Therefore, the user's data can be storedin the network or “cloud.” For example, the user can work on documentson a client device that are stored remotely on the cloud (e.g., server).Data on the client device can be synchronized with the cloud.

FIG. 2 shows an exemplary computer system (e.g., client or server) ofthe present invention. In an embodiment, a user interfaces with thesystem through a computer workstation system, such as shown in FIG. 2.FIG. 2 shows a computer system 201 that includes a monitor 203, screen205, enclosure 207 (may also be referred to as a system unit, cabinet,or case), keyboard or other human input device 209, and mouse or otherpointing device 211. Mouse 211 may have one or more buttons such asmouse buttons 213. It should be understood that the present invention isnot limited to computer workstation systems. A user can interface withany computing device, including smartphones, personal computers,laptops, electronic tablet devices, global positioning system (GPS)receivers, portable media player, personal digital assistant (PDA),other network access devices, and other processing devices capable ofreceiving or transmitting data.

In a specific implementation, the client device is a smartphone ortablet device, such as the Apple iPhone (e.g., Apple iPhone 5), AppleiPad (e.g., Apple iPad, Fourth Generation, or Apple iPad mini), SamsungGalaxy product (e.g., Samsung Galaxy S3 and Samsung Galaxy Note 2),Google Nexus devices (e.g., Google Nexus 4, Google Nexus 7, and GoogleNexus 10), and Microsoft devices (e.g., Microsoft Surface tablet).Typically, a smartphone includes a telephony portion (and associatedradios) and a computer portion, which are accessible via a touchscreendisplay. There is nonvolatile memory to store data of the telephoneportion (e.g., contacts and phone numbers) and the computer portion(e.g., application programs including a browser, pictures, games,videos, and music). The smartphone typically includes a camera (e.g.,front facing camera and rear camera) for taking pictures and video. Forexample, a smartphone or tablet can be used to take live video that canbe streamed to one or more other devices.

Enclosure 207 houses familiar computer components, some of which are notshown, such as a processor, memory, mass storage devices 217, and thelike. Mass storage devices 217 may include mass disk drives, floppydisks, magnetic disks, optical disks, magneto-optical disks, fixeddisks, hard disks, CD-ROMs, recordable CDs, DVDs, recordable DVDs (e.g.,DVD-R, DVD+R, DVD-RW, DVD+RW, HD-DVD, or Blu-ray Disc), flash and othernonvolatile solid-state storage (e.g., USB flash drive),battery-backed-up volatile memory, tape storage, reader, and othersimilar media, and combinations of these.

A computer-implemented or computer-executable version or computerprogram product of the invention may be embodied using, stored on, orassociated with computer-readable medium. A computer-readable medium mayinclude any medium that participates in providing instructions to one ormore processors for execution. Such a medium may take many formsincluding, but not limited to, nonvolatile, volatile, and transmissionmedia. Nonvolatile media includes, for example, flash memory, or opticalor magnetic disks. Volatile media includes static or dynamic memory,such as cache memory or RAM. Transmission media includes coaxial cables,copper wire, fiber optic lines, and wires arranged in a bus.Transmission media can also take the form of electromagnetic, radiofrequency, acoustic, or light waves, such as those generated duringradio wave and infrared data communications.

For example, a binary, machine-executable version, of the software ofthe present invention may be stored or reside in RAM or cache memory, oron mass storage device 217. The source code of the software of thepresent invention may also be stored or reside on mass storage device217 (e.g., hard disk, magnetic disk, tape, or CD-ROM). As a furtherexample, code of the invention may be transmitted via wires, radiowaves, or through a network such as the Internet.

FIG. 3 shows a system block diagram of computer system 201 used toexecute the software of the present invention. As in FIG. 2, computersystem 201 includes monitor 203, keyboard 209, and mass storage devices217. Computer system 501 further includes subsystems such as centralprocessor 302, system memory 304, input/output (I/O) controller 306,display adapter 308, serial or universal serial bus (USB) port 312,network interface 318, and speaker 320. The invention may also be usedwith computer systems with additional or fewer subsystems. For example,a computer system could include more than one processor 302 (i.e., amultiprocessor system) or a system may include a cache memory.

Arrows such as 322 represent the system bus architecture of computersystem 201. However, these arrows are illustrative of anyinterconnection scheme serving to link the subsystems. For example,speaker 320 could be connected to the other subsystems through a port orhave an internal direct connection to central processor 302. Theprocessor may include multiple processors or a multicore processor,which may permit parallel processing of information. Computer system 201shown in FIG. 2 is but an example of a computer system suitable for usewith the present invention. Other configurations of subsystems suitablefor use with the present invention will be readily apparent to one ofordinary skill in the art.

Computer software products may be written in any of various suitableprogramming languages, such as C, C++, C#, Pascal, Fortran, Perl, Matlab(from MathWorks, www.mathworks.com), SAS, SPSS, JavaScript, AJAX, Java,Erlang, and Ruby on Rails. The computer software product may be anindependent application with data input and data display modules.Alternatively, the computer software products may be classes that may beinstantiated as distributed objects. The computer software products mayalso be component software such as Java Beans (from Sun Microsystems) orEnterprise Java Beans (EJB from Sun Microsystems).

An operating system for the system may be one of the Microsoft Windows®family of operating systems (e.g., Windows 95, 98, Me, Windows NT,Windows 2000, Windows XP, Windows XP x64 Edition, Windows Vista, Windows7, Windows 8, Windows CE, Windows Mobile, Windows RT), Symbian OS,Tizen, Linux, HP-UX, UNIX, Sun OS, Solaris, Mac OS X, Apple iOS,Android, Alpha OS, AIX, IRIX32, or IRIX64. Other operating systems maybe used. Microsoft Windows is a trademark of Microsoft Corporation.

Furthermore, the computer may be connected to a network and mayinterface to other computers using this network. The network may be anintranet, internet, or the Internet, among others. The network may be awired network (e.g., using copper), telephone network, packet network,an optical network (e.g., using optical fiber), or a wireless network,or any combination of these. For example, data and other information maybe passed between the computer and components (or steps) of a system ofthe invention using a wireless network using a protocol such as Wi-Fi(IEEE standards 802.11, 802.11a, 802.11b, 802.11e, 802.11g, 802.11i,802.11n, 802.11 ac, and 802.11 ad, just to name a few examples), nearfield communication (NFC), radio-frequency identification (RFID), mobileor cellular wireless (e.g., 2G, 3G, 4G, 3GPP LTE, WiMAX, LTE, LTEAdvanced, Flash-OFDM, HIPERMAN, iBurst, EDGE Evolution, UMTS, UMTS-TDD,1xRDD, and EV-DO). For example, signals from a computer may betransferred, at least in part, wirelessly to components or othercomputers.

In an embodiment, with a web browser executing on a computer workstationsystem, a user accesses a system on the World Wide Web (WWW) through anetwork such as the Internet. The web browser is used to download webpages or other content in various formats including HTML, XML, text,PDF, and postscript, and may be used to upload information to otherparts of the system. The web browser may use uniform resourceidentifiers (URLs) to identify resources on the web and hypertexttransfer protocol (HTTP) in transferring files on the web. The presentsystem is not limited to run in a web browser. In other implementations,the user accesses the system through either or both of native andnonnative applications. Native applications are locally installed on theparticular computing system and are specific to the operating system orone or more hardware devices of that computing system, or a combinationof these. These applications (which are sometimes also referred to as“apps”) can be updated (e.g., periodically) via a direct internetupgrade patching mechanism or through an applications store (e.g., AppleiTunes and App store, Google Play store, and Windows Phone App store).

The system can run in platform-independent, nonnative applications. Forexample, client can access the system through a web application from oneor more servers using a network connection with the server or serversand load the web application in a web browser. For example, a webapplication can be downloaded from an application server over theInternet by a web browser. Nonnative applications can also be obtainedfrom other sources, such as a disk.

FIG. 4 shows an example of a typical live video conferencing sessionclient screen, with the main high resolution video-plane overlap withlower resolution picture-in-picture content.

FIG. 5 shows show an example of a typical multisource live newsstreaming client screen, where the main high resolution video coexistswith multiple lower resolution streams.

FIG. 6 shows a block diagram of the system elements and theirfunctionality. The system includes single or multiple content sources,weighted temporal redundancy generator, a network, and single ormultiple content consuming entities.

The single or multiple content sources can include Broadcast EncodingStations, content generating portion of Video Conferencing clients,Transcoding Gateways and Prerecorded content distribution systems. Theseare indicated as “Source” in FIG. 6.

A weighted temporal redundancy generator functions to have the abilityto provide different levels of error protection for the different partsof the processed content. The communications network is the means ofdata distribution. The single or multiple content consuming entities caninclude Retransmission nodes, Transcoding Gateways, Video ConferencingClients, and Multimedia content players. These are indicated as “Client”in FIG. 6.

Operational Principles

The system includes a set of special content generation rules at theSource in combination with a weighted temporal redundancy generator.

The source generates multiple versions of the same logical contentstreams, where a logical relation marks the content's origin whilebit-level representation is different for each version. Morespecifically, each version represents different resolution of the sameoriginal content starting with full detailed (highest resolutionintended for distribution) stream further called HR content, andfollowed by one or more lower resolution streams built by encodingscaled down versions of the original content further called LR content.Multiple LR contents would represent a nested implementation. Since atany given time there are two or more copies of the same original contentalbeit in different resolutions, this is introducing a spatialredundancy to the system.

A Weighted Temporal Redundancy (WTR) generator injects redundancy in thetemporal domain, such as a Forward Error Correction (FEC) code or aRetransmission protocol for error and packet loss protection.

HR content compared to LR content exhibits such a property that while HRcontent is the most desirable, it occupies the majority of thenetworking bandwidth and has a higher cost of error protection.Therefore, in the present invention, WTR assigns different levels oferror protection to the HR content and LR content, with LR content usingthe strongest protection while HR content using weakest protection or notemporal protection at all.

FIG. 7 shows a block diagram of content recovery and selection by theclient.

Both HR content and LR content are distributed over the networkfollowing the same path of any topology (point-to-point, or startopology through server) including unicast and multicast type oftransmission and arrives at the client.

The client is unique because it processes both HR content and LR contentsimultaneously. The client senses the stream errors and/or packet lossthrough means outside the scope of this invention (customary means inthe art), applies error recovery based on methodology utilized by theWTR generator and flags content as erroneous or intact. An FEC basedsolution may encounter error rates at excess of FEC capabilities while aretransmission protocol would run out of the latency and/or bandwidthallowance. That is a tradeoff that is a function of the desiredresolution.

FIG. 8 shows a block diagram of a specific implementation of contentreconstruction. There are multiple generated versions of the samelogical content, where each version represents a different resolution ofthe original content. The highest resolution versions of the contentstream, intended for distribution, is indicated in FIG. 8 as HR frame,in 801 a-807 a. The lower resolution versions of that same contentstream, which are built by encoding scaled down versions of the originalcontent, are indicated as LR Frame, in 801 b-807 b. At any given time,there are two or more versions of the same original content, albeit indifferent resolutions, this is introducing a spatial redundancy to thesystem.

The client outputs HR content as long as its path through the networkand delivery is intact. In case of erroneous condition with HR content804 a and 806 a, the client substitutes the portion of the content withan LR content equivalent, and output the LR content 804 c. Where the LRequivalent also contains error 806 b, the display output will exhibit anerror 806 c. Where there is no error in the HR content, the display willoutput the HR content 801 c, 802 c, 803 c, 805 c, and 807 c. This way,the user experiences the loss of detail but not the loss of the eventitself. The client falls back to traditional methods of error recoverysuch as content skipping in case all the source streams are exhibitingerrors for overlapping segments. The probability of interruption of Livestreaming content flow is minimal or zero.

Further user experience is improved by applying Super Resolutiontechniques to the LR content while substituting the HR content. Suchtechniques can include Motion Vector based SR, and reference-based SRusing HR content for the reference library of high-resolution elements.

Super Resolution

In addition to classic recovery techniques mentioned, SR can be used. Inclassic SR, subpixel offset is controlled and instead of collecting Nx(x0, x1, . . . x(N−1)) pixels at higher resolution, one collects spatialimprints of those pixels in Nx lower resolution images (y0, y1, . . . ,y(N−1)). As an example, in building a 4× resolution with 4 images, the 4LR images have a relationship with the HR image as follows:y0=(x0+x1+x2+x3)/4

This example shows simple averaging function of HR pixels falling intoLR pixel spatial domain.

By subpixel shift, one creates y1, y2, y3 in the form of:y1=(x1+x2+x3+x4)/4, yn=(xn+x(n+1)+x(n+2)+x(n+3))/4

This creates a system of solvable equations with 4x y's corresponding toeach x. Since pixels on the edges are known, the whole model unwindssimply with linear complexity.

Reference based SR can be applied to improve LR content. With HR contentavailable most of the time, a reference object database is built andused to replace LR objects with HR references. This method is known as“Image Hallucination,” since HR references are not actual HR objectsthey are representing.

For reference based SR, we need to have the ability to recognize thepatterns (scaled, rotated) to build a reference database.

Bandwidth Impact and Client Complexity

Image Plane processing is not required to generate video streams. Mainvideo-plane overlap with Picture-in-Picture content, either limits thenumber of streams multiplexed in (such as in FIG. 4), hence there willbe a minimal increase in downstream bandwidth (10-15% for 4Picture-in-Picture streams), or requires a larger final image to fit alarge amount of streams (such as in FIG. 5), and therefore doesn'tprovide bandwidth benefits vs. bit stream multiplexing approach.

Reduced networking protocol overhead (sync, headers are not repeated foreach video stream but rather multiple video streams bundled into asingle networking flow) reduces bit stream multiplexing bandwidthoverhead even further.

The Client has to only marginally increase central processing unit (CPU)power for image processing and video decoding to handle multiple imagestreams vs. a single video stream. In the case of FIG. 5, actualprocessing is lower since the total amount of pixels processed is lowercompare to enlarged video stream.

All within acceptable margin of existing HW/SW and Upstream Networkcapabilities.

Video Conferencing Application without Traditional MCU Transcoding Basedon Video Stream Multiplexing

This is a similar concept to Vidyo “Video Router” technology butinstead, uses standard video codecs without SVC Appendix and special HWor SW codec modifications.

FIG. 9 shows the internal operation of a typical state of the art videoconferencing MCU. FIG. 10 shows a specific implementation of a videoconferencing MCU operation of the present invention. Marginal complexityincrements at the client create hierarchical video streaming anddrastically reduces computational load at the MCU.

This description of the invention has been presented for the purposes ofillustration and description. It is not intended to be exhaustive or tolimit the invention to the precise form described, and manymodifications and variations are possible in light of the teachingabove. The embodiments were chosen and described in order to bestexplain the principles of the invention and its practical applications.This description will enable others skilled in the art to best utilizeand practice the invention in various embodiments and with variousmodifications as are suited to a particular use. The scope of theinvention is defined by the following claims.

The invention claimed is:
 1. A system comprising: a plurality of singleor multiple content sources; weighted temporal redundancy generator; anetwork, coupling the single or multiple content sources and weightedtemporal redundancy generator, wherein the network distributes data ofthe sources and generator; a plurality of single or multiple contentconsuming entities, coupled to the network; wherein the plurality ofsingle or multiple content sources generate multiple versions of thesame logical content-streams, where a logical relation marks thecontent's origin while bit-level representation is different for eachversion, wherein each version represents different resolution of thesame original content starting with full detailed highest resolutionintended for distribution stream further called high resolution (HR)content, and followed by one or more lower resolution streams built byencoding scaled down versions of the original content further called lowresolution (LR) content.
 2. The system of claim 1 wherein the contentsources comprise at least one of Broadcast Encoding Stations, contentgenerating portion of Video Conferencing clients, Transcoding Gateways,or Pre-recorded content distribution systems.
 3. The system of claim 1wherein a weighting property of the weighted temporal redundancygenerator is classified as the ability to provide different level oferror protection for the different parts of the processed content. 4.The system of claim 1 wherein the content consuming entities comprise atleast one of Retransmission nodes, Transcoding Gateways, VideoConferencing Clients, or Multimedia content players.
 5. The system ofclaim 4 wherein the content consuming entities are client-type devices.6. The system of claim 1 wherein a nested implementation comprisesmultiple LR content, whereby since at any given time there are two ormore copies of the same original content albeit in differentresolutions, this introduces a spatial redundancy to the system.
 7. Thesystem of claim 1 wherein the weighted temporal redundancy (WTR)generator injects redundancy in the temporal domain, comprising at leastone of a Forward Error Correction (FEC) code or a Retransmissionprotocol for error and packet loss protection.
 8. The system of claim 7wherein high resolution (HR) content is compared to low resolution (LR)content exhibits such a property that while HR content is the moredesirable, it occupies the majority of the networking bandwidth and hasa higher cost of error protection.
 9. The system of claim 7 wherein WTRassigns different levels of error protection to the HR content and LRcontent, with LR content using the strongest protection while HR contentusing weakest protection or no temporal protection at all.
 10. Thesystem of claim 1 wherein high resolution (HR) content and lowresolution (LR) content are distributed over the network using atopology including at least one of a point-to-point or star topologythrough a server, or unicast or multicast type of transmission, fordelivery to a client device.
 11. The system of claim 1 wherein theplurality of single or multiple content consuming entities processesboth high resolution (HR) content and low resolution (LR) contentsimultaneously.
 12. The system of claim 11 wherein the plurality ofsingle or multiple content consuming entities sense stream errors orpacket loss, or both, applies error recovery based on methodologyutilized by the weighted temporal redundancy generator and flags contentas erroneous or intact.
 13. The system of claim 11 wherein the pluralityof single or multiple content consuming entities output HR content aslong as its path through the network and delivery is intact.
 14. Thesystem of claim 13 wherein upon the occurrence of an erroneous conditionwith HR content, the plurality of single or multiple content consumingentities substitute a portion of the content with a LR contentequivalent.
 15. The system of claim 11 comprising applying one or moreSuper Resolution techniques to the LR content while substituting the HRcontent.
 16. The system of claim 15 wherein the Super Resolutiontechniques comprise Motion Vector based SR, Reference based SR using HRcontent for the reference library of high-resolution elements.